Papers with scheduled sampling baselines
Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to correct exposure bias in machine translation are inadequate . scheduled sampling assumes that words are aligned at each time step . |
| Approach: | They propose a differentiable sampling algorithm that optimizes the probability that the reference can be aligned with the sampled output. |
| Outcome: | The proposed approach improves BLEU on translation tasks and is simpler to train with no sampling schedule. |
Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Autoregressive models trained with maximum likelihood estimation suffer from exposure bias, i.e. the discrepancy between ground-truth prefixes used during training and model-generated prefix at inference time. |
| Approach: | They propose to use Elastic Weight Consolidation to better balance mitigating exposure bias with retaining performance. |
| Outcome: | The proposed method significantly outperforms maximum likelihood estimation and scheduled sampling baselines on four translation datasets. |